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ABSTRACT 

Every year approximately 1,300 flrst-vear students at 
the Rochester Institute of Technology complete a SO-mlnute placement 
essay during summer and fall orientations. The essays a/re scored 
hollstlcally, and the students are placed Into one of three levels of 
an English composition course* At the end of the lO'-week quarter of 
Instruction, students take an exit exsun which Is rated as either 
"passing** or Incomplete." International students take the same 
placement exams, but their essays are graded by Instructors 
Knowledgeable In English as a Second Language. The results are used 
for placement In English composition and for referrals Into second 
language instruction. Learning disabled students are allowed to take 
their exams In the most appropriate environment and may use a word 
processor. Deaf or hearing Impaired students (approximately 293 
undergraduates in academic year 1991-92 » once they are placed into 
English composition, have the option of enrolling In a section 
restricted to deaf students, or taking the course with hearing 
students and using the services of an Interpreter and notetaker. In 
1990, placement and exit exams of 120 randomly- selected freshmen were 
hollstlcally and anonymously scored. Results indicated that: (1) the 
majority of students demonstrated Improved writing skills after the 
course; (2) placement and exit tests can be accepted as valid; and 
(3) the holistic exams used can be considered reliable. (RS) 
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This talk was presented at the 1992 Conference on College Composition and 
Communication, as part of a panel titled. "Assessment Issues: General 



^1 Competency. Career Prep, and Special Student Populations." 

CO 

Q This presentation will provide an overview of the large-scale writing 



Introduction 



assessment model that is fundamental to the English Composition program 
at Rochester Institute of Technology. I will describe the administration and 
scoring of our plicement and eiit te«ti, the accommodations we 
make for special student populations (international, deaf and learning 
disabled), and the research we conducted in 1990 which compared 
students' writing before and after they had taken English Composition. 



Conposition Program 

Since the verbal abilities of incoming KIT students vary tremendously, over 
a decade ago the department of Language and Literature decided to offer 
English Composition in three levels: basic, intermediate and advanced. 
Students are placed into these levels according to their performance on a 
holistically-scored essay, and many times the student's Test of Standard 
(V) Written English (or TSWE) score is also considered when making a 

^ placement decision. In order to guarantee that students completing English 

^ Composition have a minimal level of writing competence, all students 

^ enrolled in the course - regardless of their placement level take an exit 

exam in common during the last week of class. 
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Orientation Testing 

The writing ability of students entering RIT is first assessed during our 
summer and fail orientations. Each year during the orientations, 
approximately 1300 first-year students complete a fifty-minute pli»cement 
essay on a topic requiring only general knowledge. 

Essay Scoring Procedures 

The placement essays, which I administer with the help of the orientation 
staff and a composition instructor, are scored holistically on a scale of 1 
through 6, with 1 being the lowest score. The readers are experienced 
composition faculty who teach the course and participate in practice 
scoring sessions each year. The scoring guide we use reflects the objectives 
of the course. For example, an essay with a score of 1 is described as 
having "many serious grammatical, spelling and diction errors, with poor 
organization and development of ideas." Representing the highest score on 
the placement scale, a rating of 6 is characterized as being "generally free 
of errors, having very good unity and coherence, with paragraphs 
employing an argument or multiple illustrations that are more than merely 
commonplace." The essay also demonstrates "an evident sense of style and 
perhaps, an effective use of humor or irony." 

During the scoring process, each student ess-^y is read by two different 
readers and given two scores. In general, based on those scores, the 
students are placed into the three levels of English Composition, as follows; 
Scores of 1 and 1 through 2 and 2; Basic Composition 
Scores of 3 and 3 through 4 and 4: Intermediate Composition 
Scores of 5 and 5 through 6 and 6; Advanced Composition 



However, an essay with two scores of more than one point difference is 
given a third reading, and scores of 3/3 and 4/4 are actually considered 
"borderline. " In these cases, the student's Test of Standard Written English 
score, and/or SAT verbal score, is used to help determine the appropriate 
level of placement. 

Biit Exam 

At the end of the ten-week quarter of instruction, students taking English 
Composition are given an exit exam which is administered in class by the 
instructor. In scoring the exit exam, we do not use the 1 through 6 scale 
used for placement, but instead rate the exams simply as "passing " or 
"incomplete. " Instructors do not evaluate their own students' exams; 
instead, the exams are distributed anonymously by seaion among all 
composition instructors. Each exam is then read and scored by two people 
other than the student's own professor. 

According to the scoring guide created by the faculty, a passing exit exam 
must display a logical organizational plan; attempt to develop ideas through 
details and examples; and contain only occasional errors in grammar and 
spelling. Furthermore, these errors must not seriously impede 
communication. An exit exam is rated as "incomplete " when it is 
disorganized; if it contains insufficient concrete details or examples; or if it 
contains frequent errors in grammar or spelling. Furthermore, in essays 
evaluated as "incomplete, " these errors often seriously impede 
communication. 

If a student receives a split score of one "Incomplete"' and one "Pass " on 
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the exit exam, the exam is read by a third reader. Students who receive 
two "I's" on their exam do not receive credit for the course until they pass 
a re-test. If an instructor beheves a student's score on the exam does not 
reflect the student's true ability, the instructor can immediately re-test the 
student, and the exam is scored again by two outside readers. This 
obviously prolongs the scoring process, and each person who re-tests a 
student must be willing to continue reading and scoring the re-tests of 
others. However, by allowing students and instructors the option of an 
immediate re-test, we have tried to eUminate any perception of 
arbitrariness or unfairness in the system. If a student deserves a second 
chance, the instructor has the means to provide it, and about half of those 
students who are immediately re-tested do pass. The initial failure rate on 
the exit exam varies from quarter to quarter, but is approximately 10% of 
those enrolled. The final failure rate, after the re-tests, is approximately 
5-7% . In most cases, these students sign a contract agreeing to work in the 
Writing Lab of RIT's Learning Development Center, and although they have 
two quarters in which to re-take and pass the exam, most do so within ten 
weeks. 

Accofflinodations for Special Populitions 

The ways in which the placement and exit exams are administered are 
modified for students in several special populations at RIT. Shortly after 
they arrive on campus, intcrnttionil students, of whom there are 
currently 397 undergraduates, are given the same placement exam as 
other students as part of a general assessment of their EngUsh skills in 
reading, writing and conversation. In this instance, the placement exam is 
administered along with the Michigan Test, and is scored by instructors in 



the English Language Center, instead of the composition staff. The 
instructors reading the exam are experienced in teaching EngUsh as a 
Second Language, and their evaluation of the international students' 
writing is used for referrals into ESOL instruction, as well as placement in 
English Composition. In fact, they use the same scoring guide used by 
Composition instructors, with the exception that the lower range of the 
scale is further sub-divided in order to make distinctions for ESOL 
placement. By allowing one essay to serve two purposes, we have 
eUminated redundant testing during the orientation period, maximized the 
usefulness of a single test for assessment and placement purposes, and 
mmimized the amount of staff time spent in the assessment process. Later, 
when the international students take the exit exam they may be given 
extra time to complete the exam, at the instructor 's discretion. 

Currently at RIT, approximately 170 students have been identified by the 
federally-funded Office of Special Services as having a learning disability. 
These learning disabled students, at their request or that of the Office of 
Special Services, are allowed extra time on the placement and exit exams. 
They may also take the exams privately in a quiet room rather than in the 
auditorium where placement exams are given or in the classroom where 
exit exams are given. They are allowed to use a word processor, and 
through Special Services, the students are provided proctors so that they 
may take their exams in the most appropriate environment. For some 
students, that means taking the exit exam in their dorm room, on their own 
word processor, under the supervision of a proctor. 



There are approximately 293 deaf or hard-of -hearing students 

enrolled in RIT baccalaureate programs. Through a combination of their 
scores on the Michigan Test and the California Reading Test, these students 
are identified as having English skills at one of 4 levels: preparatory, 
marginally qualified, provisionally qualified or proficient. Only at the 
"proficient" level, that is having a Michigan score of greater than or equal 
to 80 and. a California score of greater than or equal to 10, are students 
allowed to take the placement test for English Composition. Below the 
"proficient" level, "provisionally qualified" students take the placement 
exam for pre- Composition courses; marginally qualified" students 
complete 20 hours of English at the National Technical Institute for the 
Deaf and build a writing portfolio for review before being allowed to take 
the placement test for pre-Composition courses: and "preparatory" 
students, that is, those with Michigan scores equal to or below 60 anl 
California scores equal to or below 8.0, must begin their English studies 
with at least 20 credits of NTID English before moving on to the portfolio 
and then placement evaluations. 

Once they are placed in English Composition, deaf and hard-of -hearing 
students have the option of enrolling in a section restricted to deaf 
students or of taking the course with hearing students, in which case an 
interpreter and notetaker will be assigned to the class. Students in the deaf 
sections are allowed 2 hours for the exit exam, and those in non-deaf 
sections may also be given extra time, at the instruaor's discretion. 



Later in this program Di'. Jeannee Sacken, Writing Coordinator for NTID, 
will discuss how the College of Business at RIT has modified its writing 
competency exam to accommodate deaf students who are meeting 
graduation requirements for a baccalaureate degree in business. 

Research Project 1990-91 

In the summer of 1990, at the request of the Dean of the College of Liberal 
Arts. Dr. Joseph Nassar and I designed a research project to measure the 
improvement in student writing during English Composition. With the help 
of a statistics professor from the College of Science.we developed a model 
for comparing the placement and exit exams of students taking EngUsh 
Composition. Not only did the study provide valuable information 
on the improvement in student writing, it also helped us draw some 
conclusions about the validity and reUability of our approach to writing 
assessment . 

Design 

In this study, the placement and exit exams of a randomly selected group 
of 120 freshmen were hoUstically scored and rated on to a 1-6 scale by a 
group of experienced English Composition instructors. Each exam was 
scored "blindly" by two different readers. Since the compostion course is 
offered at three "levels," the study was designed so that there were equal 
numbers of students drawn from each level. Thus, we randomly selected 
40 students each from the basic, intermediate, and advanced levels of 
English Composition. Ultimately, 1 18 students participated in the study. 
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During a day-long scoring session, which included coffee and lunch breaks, 
four readers each read 12 bundles of exams (10 exams per bundle). The 
bundles had no identifying information, that is, readers did not know who 
the students were, which level of Composition they had taken, or whether 
an exam was a placement or exit test. The bundle distribution plan 
guaranteed that each student's placement and exit exams were read by the 
same two readers. For the purposes of this study, all exams (placement 
exit) were scored on the 1-6 scale we usually use for placement. Thus, each 
student writer was assigned two 2 -number scores, one pair for the 
placement exam and one pair for the exit exam. 

For example, a student might receive a 2 and 3 on the placement exam, and 
a 3 and 4 on the exit exam. Combining each pair of scores would result in a 
total score of 5 on the placement 3xam and a total score of 7 on the exit, for 
an increase in 2 points. 

Findings 

Our analysis of the comparison of 1 18 students' scores on the placement 
and exit exams indicated that 70 students, a clear majority, received 
higher scores on their exit exam than on their placement exam. There were 
24 students whose scores remained the same and also 24 whose exit exam 
was lower than their placement exam. Since variabiUty in test scores can 
be attributed to many sources, the statistical model we used attempted to 
account for differences in students, course instructors, and readers. Most 
of the variability in test scores was atti ibuted to differences in students; in 
fact, the combined differences attributed to instructors and readers was 
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less than the differences allribuled to students. Thus, the differences in 
instructors and readers could not be said to significantly affect student 
scores.The frequency and degree of change, however, did correlate with the 
students' placement level. Students enrolled in the intermediate level 
sections demonstrated the greatest frequency and degree of positive 
change, followed by the students in basic sections, and then those in 
advanced sections. 

Validity tnd Reliablity 

The findings of this research study were heartening for several reasons. 
First, as we would hope, the majority of students demonstrated improved 
writing skills after the course, when their placement and exit exams were 
compared. Second, because we used a scoring guide which was developed 
by faculty and which ranks students essays according to characteristics of 
writing we would like to see developed in the course, the placement and 
exit tests can be accepted as valid, since they assess student ability 
according to agreed-upon, course-based criteria. Finally, by demonstrating 
that differences in instructors and readers were not significant in affecting 
students' scores, the research indicated that the results could be replicated 
and that the holistic exams we use for assessing writing in English 
Composition can be considered reliable. 



I'd like to now introduce Prof. Marcia Birken, Assistant Department Hjad in 
Mathematics, from RIT's College of Science. Prof. Birken is going to discuss 
writing assessment and career preparation for mathematics students. 
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